Expose subgraph logs via subgraph GraphQL #6278

fordN · 2026-01-15T18:20:44Z

This PR introduces a more flexible subgraph log storage and querying system for Graph Node and enables subgraph logs to be queried through the GraphQL subgraph query API. The implementation supports multiple log storage backends (File, Elasticsearch, and Loki) with a consistent query interface exposed to users in the subgraph's GraphQL schema.

What's new

GraphQL Query API

New _logs query field on all subgraph deployments
Filter by log level, timestamp range, and text search
Structured log entries with metadata (source location, arguments, subgraph ID)
Support for pagination via first/skip parameters

Storage Backends

File: JSON Lines format for local development (one file per subgraph)
Elasticsearch: Enterprise search and analytics for production
Loki: Grafana's lightweight log aggregation system
Disabled: Default mode with no log storage overhead

Architecture

LogDrain: Sink for writing logs to storage backends (File, Loki, Elasticsearch)
LogStore: Read interface for querying logs from backends
LoggerFactory: Refactored to support multi-backend log routing
Configuration: Unified GRAPH_LOG_STORE_* environment variables with CLI argument parity

Examples

Querying logs

{
  _logs(
    level: ERROR
    search: "timeout"
    from: "2024-01-15T00:00:00Z"
    to: "2024-01-16T00:00:00Z"
    first: 100
  ) {
    id
    timestamp
    level
    text
    arguments {
      key
      value
    }
    meta {
      module
      line
      column
    }
  }
}

Configuring the logs store backend

File-based (development):

GRAPH_LOG_STORE_TYPE=file \
GRAPH_LOG_STORE_FILE_DIRECTORY=/var/log/subgraphs \
cargo run -p graph-node

Loki (production):

GRAPH_LOG_STORE_TYPE=loki \
GRAPH_LOG_STORE_LOKI_ENDPOINT=http://loki:3100 \
cargo run -p graph-node

Introduces the foundation for the log store system with: - LogStore trait for querying logs from backends - LogLevel enum with FromStr trait implementation - LogEntry and LogQuery types for structured log data - LogStoreFactory for creating backend instances - NoOpLogStore as default (disabled) implementation

Implements three log storage backends for querying logs: - FileLogStore: Streams JSON Lines files with bounded memory usage - ElasticsearchLogStore: Queries Elasticsearch indices with full-text search - LokiLogStore: Queries Grafana Loki using LogQL All backends implement the LogStore trait and support: - Filtering by log level, timestamp range, and text search - Pagination via first/skip parameters - Returning structured LogEntry objects Dependencies added: reqwest, serde_json for HTTP clients.

Implements slog drains for capturing and writing logs: - FileDrain: Writes logs to JSON Lines files (one file per subgraph) - LokiDrain: Writes logs to Grafana Loki via HTTP push API Both drains: - Capture structured log entries with metadata (module, line, column) - Format logs with timestamp, level, text, and arguments - Use efficient serialization with custom KVSerializers

Adds a configuration layer for selecting and configuring log backends: - LogStoreConfig enum with variants: Disabled, File, Elasticsearch, Loki - LogConfigProvider for loading config from environment variables and CLI args - Unified GRAPH_LOG_STORE_* environment variable naming - CLI arguments with --log-store-backend and backend-specific options - Configuration precedence: CLI args > env vars > defaults - Deprecation warnings for old config variables Supported configuration: - Backend selection (disabled, file, elasticsearch, loki) - File: directory, max size, retention days - Elasticsearch: endpoint, credentials, index, timeout - Loki: endpoint, tenant ID

Refactors LoggerFactory to use LogStoreConfig instead of elastic-only: - Replaced elastic_config with log_store_config parameter - Build ElasticLoggingConfig on-demand from LogStoreConfig::Elasticsearch - Support all log drain types (File, Loki, Elasticsearch) - Maintain backward compatibility with existing elastic configuration This enables the factory to create drains for any configured backend while preserving the existing component logger patterns.

Adds GraphQL API for querying subgraph logs: Schema types: - LogLevel enum (CRITICAL, ERROR, WARNING, INFO, DEBUG) - _Log_ type with id, timestamp, level, text, arguments, meta - _LogArgument_ type for structured key-value pairs - _LogMeta_ type for source location (module, line, column) Query field (_logs) with filters: - level: Filter by log level - from/to: Timestamp range (ISO 8601) - search: Text search in log messages - first/skip: Pagination (max 1000, skip max 10000)

Integrates _logs query into the GraphQL execution pipeline: Execution layer: - Execute _logs queries via log_store.query_logs() - Convert LogEntry results to GraphQL response objects - Handle log store errors gracefully Query parsing: - Recognize _logs as special query field - Build LogQuery from GraphQL arguments - Pass log_store to execution context Service wiring: - Create log store from configuration in launcher - Provide log store to GraphQL runner - Use NoOpLogStore in test environments This completes the read path from GraphQL query to log storage backend.

lutter · 2026-01-16T18:52:26Z

One thing I wonder about: should this be configured via environment variables or through graph-node.toml? I would lean towards the latter since it's a configuration that is unlikely to change often, and having it in the config file would let's us express more complicated configuration (though admittedly, right now the config is not very complex)

Adds comprehensive integration test for _logs query: Test implementation: - Deploys logs-query subgraph and waits for sync - Triggers contract events to generate logs - Queries _logs field with various filters - Verifies log entries are returned correctly - Tests filtering by level and text search

- Create graph/src/log/common.rs for common log drain functionality - SimpleKVSerializer: Concatenates KV pairs to strings - VecKVSerializer: Collects KV pairs into Vec<(String, String)> - HashMapKVSerializer: Collects KV pairs into HashMap - LogMeta: Shared metadata structure (module, line, column) - LogEntryBuilder: Builder for common log entry fields - level_to_str(): Converts slog::Level to string - create_async_logger(): Consistent async logger creation - Updated FileDrain, LokiDrain, and ElasticDrain to use the log common utilities

- include _logs in the set of special fields that bypass indexing error shortcutting when subgraph failed - add integration test to ensure _log queries return logs after subgraph failed

dwerner

Lgtm! I have a few questions/suggestions, but yolo

dwerner · 2026-01-16T19:26:47Z

graph/src/components/log_store/config.rs

+///
+/// # Returns
+/// The parsed u64 value, or the default if parsing fails or neither key is set
+pub fn read_u64_with_fallback(logger: &Logger, new_key: &str, old_key: &str, default: u64) -> u64 {


Nit: could lift the generic style of str::parse and avoid needing multiple functions, get a specialization of FromStr implementer for free.

i.e.

pub fn parse<F: FromStr>(&self) -> Result<F, F::Err> { FromStr::from_str(self) }

dwerner · 2026-01-16T19:29:32Z

graph/src/components/log_store/elasticsearch.rs

+            .client
+            .post(&url)
+            .json(&query_body)
+            .timeout(self.timeout);


👍 for timeout.

dwerner · 2026-01-16T19:30:43Z

graph/src/components/log_store/file.rs

+
+pub struct FileLogStore {
+    directory: PathBuf,
+    // TODO: Implement log rotation when file exceeds max_file_size


Todos left, not sure if intended to be implemented this pass.

dwerner · 2026-01-16T19:32:56Z

graph/src/components/log_store/file.rs

+    }
+
+    /// Parse a JSON line into a LogEntry
+    fn parse_line(&self, line: &str) -> Option<LogEntry> {


Design: idiomatically, parse methods have a Result<T, SomeErrorType> and I would expect that here but I get the practical. Worth considering a custom error type for log store? Particularly because we just ignore the line entirely it's effectively eaten if there's a parse error.

Edit: further reading later suggests there is a LogStoreError enum already. Use here?

dwerner · 2026-01-16T19:37:46Z

graph/src/components/log_store/loki.rs

+        Ok(entries)
+    }
+
+    fn parse_log_entry(


See previous comment - error enum useful here?

dwerner · 2026-01-16T19:38:47Z

graph/src/components/log_store/mod.rs

+
+use crate::prelude::DeploymentHash;
+
+#[derive(Error, Debug)]


Continuing my suggestion/question: maybe a child error enum that captures errors on retrieval/parsing?

dwerner · 2026-01-16T19:40:51Z

graph/src/components/log_store/mod.rs

+    },
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]


Suggestion: Is there a need for a custom LogLevel impl? I feel like you'd get all of this for free if you used tracing::Level including the FromStr impls.

dwerner · 2026-01-16T19:43:15Z

graph/src/log/common.rs

+}
+
+/// Converts an slog Level to a string representation
+pub fn level_to_str(level: Level) -> &'static str {


Related to last comment on custom log level type.

dwerner · 2026-01-16T19:47:11Z

graph/src/log/elastic.rs


 /// Serializes an slog log level using a serde Serializer.
-fn serialize_log_level<S>(level: &Level, serializer: S) -> Result<S::Ok, S::Error>
+fn serialize_log_level<S>(level: &str, serializer: S) -> Result<S::Ok, S::Error>


Was this previously Level coming in from the slog::* log import? Related to the log level stuff above.

fordN requested a review from dwerner January 15, 2026 18:20

fordN self-assigned this Jan 15, 2026

fordN added enhancement New feature or request area/graphql logs labels Jan 15, 2026

fordN removed the request for review from dwerner January 15, 2026 21:58

fordN force-pushed the ford/subgraph-logs-via-graphql branch from 688827a to 120d61b Compare January 16, 2026 00:10

fordN requested a review from dwerner January 16, 2026 00:12

fordN added 7 commits January 15, 2026 17:02

fordN force-pushed the ford/subgraph-logs-via-graphql branch from 120d61b to ee0f228 Compare January 16, 2026 01:31

fordN requested review from lutter and removed request for lutter January 16, 2026 02:04

fordN force-pushed the ford/subgraph-logs-via-graphql branch from a4432ca to 384bf35 Compare January 16, 2026 17:39

fordN added 6 commits January 16, 2026 11:42

docs: Add log store documentation

88b5419

graphql,tests: Prevent combining _logs with entity queries

8d033db

graphql, tests: Ensure logs graphql queries work on failed subgraphs

677342b

- include _logs in the set of special fields that bypass indexing error shortcutting when subgraph failed - add integration test to ensure _log queries return logs after subgraph failed

graph: Add Default impls for log serializers

881e55a

fordN force-pushed the ford/subgraph-logs-via-graphql branch from 384bf35 to 881e55a Compare January 16, 2026 19:53

dwerner reviewed Jan 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Expose subgraph logs via subgraph GraphQL #6278

Expose subgraph logs via subgraph GraphQL #6278

fordN commented Jan 15, 2026 •

edited

Loading

Uh oh!

lutter commented Jan 16, 2026

Uh oh!

dwerner left a comment

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

dwerner Jan 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Expose subgraph logs via subgraph GraphQL #6278

Are you sure you want to change the base?

Expose subgraph logs via subgraph GraphQL #6278

Conversation

fordN commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's new

Examples

Querying logs

Configuring the logs store backend

Uh oh!

lutter commented Jan 16, 2026

Uh oh!

dwerner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fordN commented Jan 15, 2026 •

edited

Loading